Clone MCMC: Parallel High-Dimensional Gaussian Gibbs Sampling
نویسندگان
چکیده
We propose a generalized Gibbs sampler algorithm for obtaining samples approximately distributed from a high-dimensional Gaussian distribution. Similarly to Hogwild methods, our approach does not target the original Gaussian distribution of interest, but an approximation to it. Contrary to Hogwild methods, a single parameter allows us to trade bias for variance. We show empirically that our method is very flexible and performs well compared to Hogwild-type algorithms.
منابع مشابه
MCMC and Naive Parallel Gibbs Sampling
In this scribe, we are going to review the Parallel Monte Carlo Markov Chain (MCMC) method. First, we will recap of MCMC methods, particularly the Metropolis-Hasting and Gibbs Sampling algorithms. Then we will show the drawbacks of these classical MCMC methods as well as the Naive Parallel Gibbs Sampling approach. Finally, we will come up with the Sequential Monte Carlo and Parallel Inference f...
متن کاملAsynchronous Distributed Gibbs Sampling (Preprint Version 0.1)
Gibbs sampling is a Markov Chain Monte Carlo (MCMC) method for numerically approximating integrals of interest in Bayesian statistics and other mathematical sciences. Since MCMC methods typically suffer from poor scaling when the integral in question is high-dimensional (for example, in problems in Bayesian statistics involving large data sets), researchers have attempted to find ways to speed ...
متن کاملThe Recycling Gibbs sampler for efficient learning
Monte Carlo methods are essential tools for Bayesian inference. Gibbs sampling is a well-known Markov chain Monte Carlo (MCMC) algorithm, extensively used in signal processing, machine learning, and statistics, employed to draw samples from complicated high-dimensional posterior distributions. The key point for the successful application of the Gibbs sampler is the ability to draw efficiently s...
متن کاملA Bayesian Prediction using the Elliptical and the Skew Gaussian Processes
A Bayesian Prediction using the Elliptical Processes (EP) and the Skew Gaussian Processes (SGP) is proposed, motivated by a Bayesian model for heavy, light tailed or skewed real data. We define weak third order stationary for the Skew Gaussian Processes. Sometimes the family of distributions have dimensional coherency (consistency) property which is important for prediction. We use a Markov Cha...
متن کاملGPU-accelerated Gibbs Sampling: a case study of the Horseshoe Probit model
Gibbs sampling is a widely used Markov Chain Monte Carlo (MCMC) method for numerically approximating integrals of interest in Bayesian statistics and other mathematical sciences. Many implementations of MCMC methods do not extend easily to parallel computing environments, as their inherently sequential nature incurs a large synchronization cost. In this paper, we show how to do Gibbs sampling i...
متن کامل